A few semesters ago, I started noticing something odd in my own course. Students would breeze through their coursework with AI tools at their fingertips, scoring at full competency, no red flags anywhere. Then the same students would sit down for a proctored exam covering the exact same material, no AI allowed, and struggle. Not a little. A lot.
At first I chalked it up to test anxiety, or maybe just an off day. But it kept happening. Different students, same pattern, semester after semester. Eventually I had to stop treating it as a fluke and start treating it as a real problem worth studying. I ended up building my entire doctoral research around it.
I call it the competency illusion, and the mechanism behind it comes down to something researchers call cognitive load. There are a few different kinds of mental effort involved in learning anything. Some of it is just friction, confusing instructions, clunky tools, stuff that gets in the way and should be minimized. But some of it is productive struggle, the actual mental work of building understanding. You cannot skip that part and still end up with real learning on the other side.
Here is the uncomfortable part. AI tools are great at removing the first kind of friction. But they can just as easily remove the second kind too, the productive struggle, without anyone noticing it happened. A student pastes in a prompt, gets a polished answer, and walks away feeling like they understood the material. They did the task. They just did not do the thinking.
That is what makes this tricky to catch. The student is not trying to cheat. They genuinely feel confident. The performance looks real, right up until the AI gets taken away.
So what do you do about it, short of just banning AI from classrooms entirely, which would throw out everything good about these tools along with the problem? My research landed on a different answer. Instead of stripping access away, build in a few specific habits: short reflection moments where students check what they actually understand versus what they think they understand, assessments that ask students to show their reasoning instead of just their final answer, and direct instruction on the difference between using AI to think alongside you versus using it to think for you.
None of this is about distrusting AI in education. I do not think that is the right lesson here. It is about making sure the confidence a student walks away with is actually earned, so that when the AI is not there anymore, the understanding still is.